On Improving the Performance of Data Partitioning Oriented Parallel Irregular Reductions

Different parallelization techniques for reductions have been proposed elsewhere, that we have classified in this paper into two classes: LPO (Loop Partitioning Oriented techniques) and DPO (Data Partitioning Oriented techniques). We have analyzed both classes in terms of a set of performance properties: data locality, memory overhead, parallelism and workload balancing. In this paper we propose several techniques to increase the exploited parallelism and to introduce load balancing into a DPO method. Regarding parallelism, the solution is based on the partial expansion of the reduction array. For load balance, a first technique is generic, as it can deal with any kind of load unbalancing present in the problem domain. A second technique handles a special case of load unbalancing, appearing when there are a large number of write operations on small regions of the reduction arrays. Efficient implementations of the proposed optimizing solutions for the DWA–LIP DPO method are presented, experimentally tested on static and dynamic kernel codes and compared with other parallel reduction methods.